Music-Induced Emotion Recognition Based on Feature Reduction Using PCA From EEG Signals
-
Hamid Khabiri
,
Mohammad Naseh Talebi
,
Mehdi Fakhimi Kamran
,
Shadi Akbari
,
Farzaneh Zarrin
,
Fatemeh Mohandesi
Abstract
Purpose: Listening to music has a great impact on people's emotions and would change brain activity. In other words, music-induced emotions are trackable in electrical brain activities. Therefore, Electroencephalography can be a suitable tool to detect these induced emotions. The present study attempted to use electroencephalography in order to recognize four types of emotions (happy, relaxing, stressful, and sad) induced in response to listening to music excerpts, using three classifiers.
Methods: In this empirical study, electroencephalography signals were collected from 20 participants, as they were listening to pieces of selected music... The collected data was then pre-processed, and 28 linear and nonlinear features for recognizing the aforementioned emotions were extracted. Feature-space components were then reduced through a principal components analysis. Finally, the first ten components of feature-space were used as input for classifiers to identify the induced emotions.
Results: The outputs showed that the suggested method was well capable of emotion recognition. Evaluating the music excerpts, on the self-assessment manikin scale, demonstrated that the labelling of the music tracks was accurate. The highest accuracy found among neural network, K-nearest neighbors, and support vector machine algorithms was respectively %84, %84, and %89 for happy emotions.
Conclusion: Reduction of features via principal components analysis, led to an acceptable accuracy in classification. Happiness was the most recognizable emotion and the support vector machine had the highest performance among the classifiers. In the end, the outcomes of the proposed method demonstrate that this system is better than the several research in EEG-based emotion recognition.
1- Seyed Ebrahim Hosseini and Seyed Ali Hosseini, "Therapeutic effects of music: A review." Report of Health Care, Vol. 4 (No. 4), pp. 1-13, (2018).
2- Antonietta Provenza, "Aristoxenus and music therapy: fr. 26 Wehrli within the tradition on music and catharsis." in Aristoxenus of Tarentum: Routledge, (2017), pp. 91-128.
3- Fachner J, Gold C, Erkkilä J. "Music therapy modulates fronto-temporal activity in rest-EEG in depressed clients." Brain Topogr;26(2):338-54, (2013). doi: 10.1007/s10548-012-0254-x. Epub 2012 Sep 16. PMID: 22983820.
4- Rosalind W. Picard, Elias Vyzas, and Jennifer Healey, "Toward machine emotional intelligence: Analysis of affective physiological state." IEEE transactions on pattern analysis and machine intelligence, Vol. 23 (No. 10), pp. 1175-91, (2001).
5- Filipe Galvão, Soraia M Alarcão, and Manuel J Fonseca, "Predicting exact valence and arousal values from EEG." Sensors, Vol. 21 (No. 10), p. 3414, (2021).
6- Mahdi Khezri and Mohammad Firoozabadi, "Adaptive Fusion of Forehead and Physiological Signals upon Emotion Recognition." Advances in Cognitive Science, Vol. 17 (No. 4), pp. 45-62, (2016).
7- Morteza Zangeneh Soroush, Keivan Maghooli, Seyed Kamaledin Setarehdan, and Ali Motie Nasrabadi, "A novel method of eeg-based emotion recognition using nonlinear features variability and Dempster–Shafer theory." Biomedical Engineering: Applications, Basis and Communications, Vol. 30 (No. 04), p. 1850026, (2018).
8- Paul Ed Ekman and Richard J Davidson, The nature of emotion: Fundamental questions. Oxford University Press, (1994).
9- Ian Daly, Duncan Williams, Faustina Hwang, Alexis Kirke, Eduardo R Miranda, and Slawomir J Nasuto, "Electroencephalography reflects the activity of sub-cortical brain regions during approach-withdrawal behaviour while listening to music." Scientific reports, Vol. 9 (No. 1), pp. 1-22, (2019).
10- Md Mustafizur Rahman et al., "Recognition of human emotions using EEG signals: A review." Computers in Biology and Medicine, Vol. 136p. 104696, (2021).
11- Baoquan Cheng, Chaojie Fan, Hanliang Fu, Jianling Huang, Huihua Chen, and Xiaowei Luo, "Measuring and computing cognitive statuses of construction workers based on electroencephalogram: a critical review." IEEE Transactions on Computational Social Systems, (2022).
12- Raja Majid Mehmood, Ruoyu Du, and Hyo Jong Lee, "Optimal feature selection and deep learning ensembles method for emotion recognition from human brain EEG sensors." Ieee Access, Vol. 5pp. 14797-806, (2017).
13- Panagiotis C Petrantonakis and Leontios J Hadjileontiadis, "A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition." IEEE Transactions on information technology in biomedicine, Vol. 15 (No. 5), pp. 737-46, (2011).
14- Xiao-Wei Wang, Dan Nie, and Bao-Liang Lu, "Emotional state classification from EEG data using machine learning approach." Neurocomputing, Vol. 129pp. 94-106, (2014).
15- Yuan-Pin Lin et al., "EEG-based emotion recognition in music listening." IEEE Transactions on Biomedical Engineering, Vol. 57 (No. 7), pp. 1798-806, (2010).
16- Wei-Long Zheng and Bao-Liang Lu, "Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks." IEEE Transactions on autonomous mental development, Vol. 7 (No. 3), pp. 162-75, (2015).
17- Jing Chen, Bin Hu, Philip Moore, Xiaowei Zhang, and Xu Ma, "Electroencephalogram-based emotion assessment system using ontology and data mining techniques." Applied Soft Computing, Vol. 30pp. 663-74, (2015).
18- Ahmet Mert and Aydin Akan, "Emotion recognition from EEG signals by using multivariate empirical mode decomposition." Pattern Analysis and Applications, Vol. 21 (No. 1), pp. 81-89, (2018).
19- Vipin Gupta, Mayur Dahyabhai Chopda, and Ram Bilas Pachori, "Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals." IEEE Sensors Journal, Vol. 19 (No. 6), pp. 2266-74, (2018).
20- Pascal Ackermann, Christian Kohlschein, Jó Agila Bitsch, Klaus Wehrle, and Sabina Jeschke, "EEG-based automatic emotion recognition: Feature extraction, selection and classification methods." in 2016 IEEE 18th international conference on e-health networking, applications and services (Healthcom), (2016): IEEE, pp. 1-6.
21- Sander Koelstra et al., "Deap: A database for emotion analysis; using physiological signals." IEEE transactions on affective computing, Vol. 3 (No. 1), pp. 18-31, (2011).
22- Panagiotis C Petrantonakis and Leontios J Hadjileontiadis, "Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis." IEEE transactions on affective computing, Vol. 1 (No. 2), pp. 81-97, (2010).
23- Yisi Liu, Olga Sourina, and Minh Khoa Nguyen, "Real-time EEG-based emotion recognition and its applications." in Transactions on computational science XII: Springer, (2011), pp. 256-77.
24- M Murugappan and Subbulakshmi Murugappan, "Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT)." in 2013 IEEE 9th International Colloquium on Signal Processing and its Applications, (2013): IEEE, pp. 289-94.
25- Jiahui Cai, Wei Chen, and Zhong Yin, "Multiple transferable recursive feature elimination technique for emotion recognition based on EEG signals." Symmetry, Vol. 11 (No. 5), p. 683, (2019).
26- Juan Abdon Miranda-Correa, Mojtaba Khomami Abadi, Nicu Sebe, and Ioannis Patras, "Amigos: A dataset for affect, personality and mood research on individuals and groups." IEEE transactions on affective computing, Vol. 12 (No. 2), pp. 479-93, (2018).
27- Stamos Katsigiannis and Naeem Ramzan, "DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices." IEEE journal of biomedical and health informatics, Vol. 22 (No. 1), pp. 98-107, (2017).
28- Mo Chen, Junwei Han, Lei Guo, Jiahui Wang, and Ioannis Patras, "Identifying valence and arousal levels via connectivity between EEG channels." in 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), (2015): IEEE, pp. 63-69.
29- John Atkinson and Daniel Campos, "Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers." Expert Systems with Applications, Vol. 47pp. 35-41, (2016).
30- Zeynab Mohammadi, Javad Frounchi, and Mahmood Amiri, "Wavelet-based emotion recognition system using EEG signal." Neural Computing and Applications, Vol. 28 (No. 8), pp. 1985-90, (2017).
31- Margaret M Bradley and Peter J Lang, "The International Affective Digitized Sounds (; IADS-2): Affective ratings of sounds and instruction manual." University of Florida, Gainesville, FL, Tech. Rep. B-3, (2007).
32- Margaret M Bradley and Peter J Lang, "Measuring emotion: the self-assessment manikin and the semantic differential." Journal of behavior therapy and experimental psychiatry, Vol. 25 (No. 1), pp. 49-59, (1994).
33- Laura J Julian, "Measures of anxiety." Arthritis care & research, Vol. 63 (No. 0 11), (2011).
34- Li Hu and Zhiguo Zhang, EEG signal processing and feature extraction. Springer, (2019).
35- Yong-Jin Liu, Minjing Yu, Guozhen Zhao, Jinjing Song, Yan Ge, and Yuanchun Shi, "Real-time movie-induced discrete emotion recognition from EEG signals." IEEE transactions on affective computing, Vol. 9 (No. 4), pp. 550-62, (2017).
36- Muhammad Najam Dar, Amna Rahim, Muhammad Usman Akram, Sajid Gul Khawaja, and Aqsa Rahim, "YAAD: Young Adult’s Affective Data Using Wearable ECG and GSR sensors." in 2022 2nd International Conference on Digital Futures and Transformative Technologies (ICoDT2), (2022): IEEE, pp. 1-7.
37- Jessica Sharmin Rahman, Tom Gedeon, Sabrina Caldwell, and Richard Jones, "Brain melody informatics: analysing effects of music on brainwave patterns." in 2020 international joint conference on neural networks (ijcnn), (2020): IEEE, pp. 1-8.
38- Ron Kohavi and George H John, "Wrappers for feature subset selection." Artificial intelligence, Vol. 97 (No. 1-2), pp. 273-324, (1997).
39- Lindsay I Smith, "A tutorial on principal components analysis." (2002).
40- Krishna Bairavi, "EEG Based Emotion Recognition System for Special Children." International Conference on Communication Engineering and Technology (ICCET '18), (2018).
41- M Zangeneh Soroush, K Maghooli, SK Setarehdan, and A Motie Nasrabadi, "Emotion classification through nonlinear EEG analysis using machine learning methods." Int. Clin. Neurosci. J, Vol. 5 (No. 4), pp. 135-49, (2018).
42- A. Bhattacharyya, R. K. Tripathy, L. Garg, and R. B. Pachori, "A Novel Multivariate-Multiscale Approach for Computing EEG Spectral and Temporal Complexity for Human Emotion Recognition." IEEE Sensors Journal, Vol. 21 (No. 3), pp. 3579-91, (2021).
43- W. L. Zheng and B. L. Lu, "Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks." IEEE Transactions on autonomous mental development, Vol. 7 (No. 3), pp. 162-75, (2015).
44- Pan Jiahui, Li Yuanqing, and Wang Jun, "An EEG-Based brain-computer interface for emotion recognition." in 2016 International Joint Conference on Neural Networks (IJCNN), (2016), pp. 2063-67.
45- Didar Dadebayev, Wei Wei Goh, and Ee Xion Tan, "EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques." Journal of King Saud University-Computer and Information Sciences, (2021).
2- Antonietta Provenza, "Aristoxenus and music therapy: fr. 26 Wehrli within the tradition on music and catharsis." in Aristoxenus of Tarentum: Routledge, (2017), pp. 91-128.
3- Fachner J, Gold C, Erkkilä J. "Music therapy modulates fronto-temporal activity in rest-EEG in depressed clients." Brain Topogr;26(2):338-54, (2013). doi: 10.1007/s10548-012-0254-x. Epub 2012 Sep 16. PMID: 22983820.
4- Rosalind W. Picard, Elias Vyzas, and Jennifer Healey, "Toward machine emotional intelligence: Analysis of affective physiological state." IEEE transactions on pattern analysis and machine intelligence, Vol. 23 (No. 10), pp. 1175-91, (2001).
5- Filipe Galvão, Soraia M Alarcão, and Manuel J Fonseca, "Predicting exact valence and arousal values from EEG." Sensors, Vol. 21 (No. 10), p. 3414, (2021).
6- Mahdi Khezri and Mohammad Firoozabadi, "Adaptive Fusion of Forehead and Physiological Signals upon Emotion Recognition." Advances in Cognitive Science, Vol. 17 (No. 4), pp. 45-62, (2016).
7- Morteza Zangeneh Soroush, Keivan Maghooli, Seyed Kamaledin Setarehdan, and Ali Motie Nasrabadi, "A novel method of eeg-based emotion recognition using nonlinear features variability and Dempster–Shafer theory." Biomedical Engineering: Applications, Basis and Communications, Vol. 30 (No. 04), p. 1850026, (2018).
8- Paul Ed Ekman and Richard J Davidson, The nature of emotion: Fundamental questions. Oxford University Press, (1994).
9- Ian Daly, Duncan Williams, Faustina Hwang, Alexis Kirke, Eduardo R Miranda, and Slawomir J Nasuto, "Electroencephalography reflects the activity of sub-cortical brain regions during approach-withdrawal behaviour while listening to music." Scientific reports, Vol. 9 (No. 1), pp. 1-22, (2019).
10- Md Mustafizur Rahman et al., "Recognition of human emotions using EEG signals: A review." Computers in Biology and Medicine, Vol. 136p. 104696, (2021).
11- Baoquan Cheng, Chaojie Fan, Hanliang Fu, Jianling Huang, Huihua Chen, and Xiaowei Luo, "Measuring and computing cognitive statuses of construction workers based on electroencephalogram: a critical review." IEEE Transactions on Computational Social Systems, (2022).
12- Raja Majid Mehmood, Ruoyu Du, and Hyo Jong Lee, "Optimal feature selection and deep learning ensembles method for emotion recognition from human brain EEG sensors." Ieee Access, Vol. 5pp. 14797-806, (2017).
13- Panagiotis C Petrantonakis and Leontios J Hadjileontiadis, "A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition." IEEE Transactions on information technology in biomedicine, Vol. 15 (No. 5), pp. 737-46, (2011).
14- Xiao-Wei Wang, Dan Nie, and Bao-Liang Lu, "Emotional state classification from EEG data using machine learning approach." Neurocomputing, Vol. 129pp. 94-106, (2014).
15- Yuan-Pin Lin et al., "EEG-based emotion recognition in music listening." IEEE Transactions on Biomedical Engineering, Vol. 57 (No. 7), pp. 1798-806, (2010).
16- Wei-Long Zheng and Bao-Liang Lu, "Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks." IEEE Transactions on autonomous mental development, Vol. 7 (No. 3), pp. 162-75, (2015).
17- Jing Chen, Bin Hu, Philip Moore, Xiaowei Zhang, and Xu Ma, "Electroencephalogram-based emotion assessment system using ontology and data mining techniques." Applied Soft Computing, Vol. 30pp. 663-74, (2015).
18- Ahmet Mert and Aydin Akan, "Emotion recognition from EEG signals by using multivariate empirical mode decomposition." Pattern Analysis and Applications, Vol. 21 (No. 1), pp. 81-89, (2018).
19- Vipin Gupta, Mayur Dahyabhai Chopda, and Ram Bilas Pachori, "Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals." IEEE Sensors Journal, Vol. 19 (No. 6), pp. 2266-74, (2018).
20- Pascal Ackermann, Christian Kohlschein, Jó Agila Bitsch, Klaus Wehrle, and Sabina Jeschke, "EEG-based automatic emotion recognition: Feature extraction, selection and classification methods." in 2016 IEEE 18th international conference on e-health networking, applications and services (Healthcom), (2016): IEEE, pp. 1-6.
21- Sander Koelstra et al., "Deap: A database for emotion analysis; using physiological signals." IEEE transactions on affective computing, Vol. 3 (No. 1), pp. 18-31, (2011).
22- Panagiotis C Petrantonakis and Leontios J Hadjileontiadis, "Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis." IEEE transactions on affective computing, Vol. 1 (No. 2), pp. 81-97, (2010).
23- Yisi Liu, Olga Sourina, and Minh Khoa Nguyen, "Real-time EEG-based emotion recognition and its applications." in Transactions on computational science XII: Springer, (2011), pp. 256-77.
24- M Murugappan and Subbulakshmi Murugappan, "Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT)." in 2013 IEEE 9th International Colloquium on Signal Processing and its Applications, (2013): IEEE, pp. 289-94.
25- Jiahui Cai, Wei Chen, and Zhong Yin, "Multiple transferable recursive feature elimination technique for emotion recognition based on EEG signals." Symmetry, Vol. 11 (No. 5), p. 683, (2019).
26- Juan Abdon Miranda-Correa, Mojtaba Khomami Abadi, Nicu Sebe, and Ioannis Patras, "Amigos: A dataset for affect, personality and mood research on individuals and groups." IEEE transactions on affective computing, Vol. 12 (No. 2), pp. 479-93, (2018).
27- Stamos Katsigiannis and Naeem Ramzan, "DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices." IEEE journal of biomedical and health informatics, Vol. 22 (No. 1), pp. 98-107, (2017).
28- Mo Chen, Junwei Han, Lei Guo, Jiahui Wang, and Ioannis Patras, "Identifying valence and arousal levels via connectivity between EEG channels." in 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), (2015): IEEE, pp. 63-69.
29- John Atkinson and Daniel Campos, "Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers." Expert Systems with Applications, Vol. 47pp. 35-41, (2016).
30- Zeynab Mohammadi, Javad Frounchi, and Mahmood Amiri, "Wavelet-based emotion recognition system using EEG signal." Neural Computing and Applications, Vol. 28 (No. 8), pp. 1985-90, (2017).
31- Margaret M Bradley and Peter J Lang, "The International Affective Digitized Sounds (; IADS-2): Affective ratings of sounds and instruction manual." University of Florida, Gainesville, FL, Tech. Rep. B-3, (2007).
32- Margaret M Bradley and Peter J Lang, "Measuring emotion: the self-assessment manikin and the semantic differential." Journal of behavior therapy and experimental psychiatry, Vol. 25 (No. 1), pp. 49-59, (1994).
33- Laura J Julian, "Measures of anxiety." Arthritis care & research, Vol. 63 (No. 0 11), (2011).
34- Li Hu and Zhiguo Zhang, EEG signal processing and feature extraction. Springer, (2019).
35- Yong-Jin Liu, Minjing Yu, Guozhen Zhao, Jinjing Song, Yan Ge, and Yuanchun Shi, "Real-time movie-induced discrete emotion recognition from EEG signals." IEEE transactions on affective computing, Vol. 9 (No. 4), pp. 550-62, (2017).
36- Muhammad Najam Dar, Amna Rahim, Muhammad Usman Akram, Sajid Gul Khawaja, and Aqsa Rahim, "YAAD: Young Adult’s Affective Data Using Wearable ECG and GSR sensors." in 2022 2nd International Conference on Digital Futures and Transformative Technologies (ICoDT2), (2022): IEEE, pp. 1-7.
37- Jessica Sharmin Rahman, Tom Gedeon, Sabrina Caldwell, and Richard Jones, "Brain melody informatics: analysing effects of music on brainwave patterns." in 2020 international joint conference on neural networks (ijcnn), (2020): IEEE, pp. 1-8.
38- Ron Kohavi and George H John, "Wrappers for feature subset selection." Artificial intelligence, Vol. 97 (No. 1-2), pp. 273-324, (1997).
39- Lindsay I Smith, "A tutorial on principal components analysis." (2002).
40- Krishna Bairavi, "EEG Based Emotion Recognition System for Special Children." International Conference on Communication Engineering and Technology (ICCET '18), (2018).
41- M Zangeneh Soroush, K Maghooli, SK Setarehdan, and A Motie Nasrabadi, "Emotion classification through nonlinear EEG analysis using machine learning methods." Int. Clin. Neurosci. J, Vol. 5 (No. 4), pp. 135-49, (2018).
42- A. Bhattacharyya, R. K. Tripathy, L. Garg, and R. B. Pachori, "A Novel Multivariate-Multiscale Approach for Computing EEG Spectral and Temporal Complexity for Human Emotion Recognition." IEEE Sensors Journal, Vol. 21 (No. 3), pp. 3579-91, (2021).
43- W. L. Zheng and B. L. Lu, "Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks." IEEE Transactions on autonomous mental development, Vol. 7 (No. 3), pp. 162-75, (2015).
44- Pan Jiahui, Li Yuanqing, and Wang Jun, "An EEG-Based brain-computer interface for emotion recognition." in 2016 International Joint Conference on Neural Networks (IJCNN), (2016), pp. 2063-67.
45- Didar Dadebayev, Wei Wei Goh, and Ee Xion Tan, "EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques." Journal of King Saud University-Computer and Information Sciences, (2021).
| Files | ||
| Issue | Vol 11 No 1 (2024) | |
| Section | Original Article(s) | |
| DOI | https://doi.org/10.18502/fbt.v11i1.14512 | |
| Keywords | ||
| Emotion Recognition Electroencephalography Principal Component Analysis Classification Music | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Khabiri H, Talebi MN, Fakhimi Kamran M, Akbari S, Zarrin F, Mohandesi F. Music-Induced Emotion Recognition Based on Feature Reduction Using PCA From EEG Signals. Frontiers Biomed Technol. 2023;11(1):59-68.
