A Multilevel Feature-based Emotion Classification Technique using Electroencephalogram Signals
Abstract
Emotion dysfunction can be an indication of mental health conditions like depression, stress, anxiety, etc. Therefore, appropriate recognition of human emotion is crucial for early monitoring of mental well-being. This study aims to develop a robust multilevel feature concatenation framework that effectively integrates handcrafted and deep learning (DL) features to improve electroencephalogram (EEG)-based emotion classification. The proposed framework is evaluated on two publicly available benchmark EEG datasets: DEAP and DREAMER. Initially, handcrafted features extracted from the statistical, time, frequency, and nonlinear domains are categorized into low-level features (LLF) and mid-level features (MLF). To select the most relevant handcrafted features, statistical analysis, minimum Redundancy Maximum Relevance (mRMR) and the Pearson correlation coefficient (PCC) are employed. In parallel, a two-dimensional convolutional neural network (2D-CNN) is used to learn hierarchical deep features, which are grouped into LLF, MLF, and high-level features (HLF). The handcrafted LLF and MLF are then concatenated with the corresponding DL-based LLF, MLF, and HLF to construct a discriminative multilevel feature representation. The resulting feature vector is classified using Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and XGBoost classifiers under a 10-fold and leave-one-subject-out (LOSO) cross-validation protocol. Performance is evaluated using accuracy, precision, recall, F1-score, and balanced accuracy. Experimental results demonstrate that the proposed framework consistently outperforms existing state-of-the-art methods. Among the evaluated classifiers, SVM achieves the highest performance, yielding accuracies of 93.07% and 98.70% for 4-class and 2-class emotion classification, respectively, on the DEAP dataset, and 93.85% and 98.78% for 4-class and 2-class emotion classification, on the DREAMER dataset. These results demonstrate that concatenation of handcrafted with DL-based features at different abstraction level significantly enhances feature discriminability and classification robustness. The proposed framework provides an effective solution for EEG-based emotion recognition, with promising applications in affective computing, human–computer interaction, and intelligent mental health monitoring.
Downloads
References
M. Jafari, A. Shoeibi, M. Khodatars, S. Bagherzadeh, A. Shalbaf, D. L. Garcia, J. M. Gorriz, and U. R. Acharya, “Emotion recognition in EEG signals using deep learning methods: A review,” Computers in Biology and Medicine, vol. 165, pp. 107450, 2023, Elsevier, https://doi.org/10.1016/j.compbiomed.2023.107450.
E. H. Houssein, A. Hammad, and A. A. Ali, “Human emotion recognition from EEG-based brain–computer interface using machine learning: A comprehensive review,” Neural Computing and Applications, vol. 34, no. 15, pp. 12527–12557, 2022, Springer, https://doi.org/10.1007/s00521-022-07292-4.
R. Sharma, R. B. Pachori, and P. Sircar, “Automated emotion recognition based on higher order statistics and deep learning algorithm,” Biomedical Signal Processing and Control, vol. 58, pp. 101867, 2020, Elsevier, https://doi.org/10.1016/j.bspc.2020.101867.
A. Nalwaya, K. Das, and R. B. Pachori, “An automated framework for human emotion detection from multichannel EEG signals,” IEEE Sensors Journal, vol. 24, no. 13, pp. 20920-20927, IEEE, 2024, https://doi.org/10.1109/JSEN.2024.3398050.
M. Khateeb, S. M. Anwar, and M. Alnowami, “Multidomain feature fusion for emotion classification using DEAP dataset,” IEEE Access, vol. 9, pp. 12134-12142, IEEE, 2021, https://doi.org/10.1109/ACCESS.2021.3051281.
K. P. Wagh and K. Vasanth, “Performance evaluation of multi-channel electroencephalogram signal (EEG)-based time frequency analysis for human emotion recognition,” Biomedical Signal Processing and Control, vol. 78, pp. 103966, Elsevier, 2022, https://doi.org/10.1016/j.bspc.2022.103966.
V. Bajaj and R. B. Pachori, “Human emotion classification from EEG signals using multiwavelet transform,” in Proceedings of the 2014 International Conference on Medical Biometrics, IEEE, pp. 125–130, 2014, https://doi.org/10.1109/ICMB.2014.29.
A. Anuragi, D. S. Sisodia, and R. B. Pachori, “EEG-based cross-subject emotion recognition using Fourier–Bessel series expansion based empirical wavelet transform and NCA feature selection method,” Information Sciences, vol. 610, pp. 508–524, Elsevier, 2022, https://doi.org/10.1016/j.ins.2022.07.121.
V. Gupta, M. D. Chopda, and R. B. Pachori, “Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals,” IEEE Sensors Journal, vol. 19, no. 6, pp. 2266–2274, 2019, IEEE, 10.1109/JSEN.2018.2883497.
S. Sheykhivand, Z. Mousavi, T. Y. Rezaii, and A. Farzamnia, “Recognizing emotions evoked by music using CNN-LSTM networks on EEG signals,” IEEE Access, vol. 8, pp. 139332–139345, IEEE, 2020, https://doi.org/10.1109/ACCESS.2020.301188.
Z. Gao, R. Li, C. Ma, L. Rui, and X. Sun, “Core-brain network-based multilayer convolutional neural network for emotion recognition,” IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1–9, IEEE, 2021, https://doi.org/10.1109/TIM.2021.3090164.
W. Tao, C. Li, R. Song, J. Cheng, Y. Liu, F. Wan, and X. Chen, “EEG-based emotion recognition via channel-wise attention and self attention," IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 382–393, IEEE, 2020,https://doi.org/10.1109/TAFFC.2020.3025777.
N. Piedrabuena, W. M. Elmannai and Y. Wang, “Emotion Recognition From EEG Signals Using a Hybrid Deep Learning Approach,” IEEE Access, vol. 14, pp. 11035-11047, IEEE, 2025, https://doi.org/10.1016/j.bspc.2024.106620.
S. Wang, X. Zhang, and R. Zhao, “Lightweight CNN-CBAM BiLSTM EEG emotion recognition based on multiband DE features,” Biomedical Signal Processing and Control, vol. 103, pp. 107435, Elsevier, 2026, https://doi.org/ 10.1109/ACCESS.2025.3642249.
E. Nsugbe, O. W. Samuel, M. G. Asogbon, and G. Li, “Intelligence combiner: A combination of deep learning and handcrafted features for an adolescent psychosis prediction using EEG signals,” in Proceedings of the 2022 IEEE International Workshop on Metrology for Industry 4.0 and IoT, pp. 92–97, IEEE, 2022, https://doi.org/10.1109/MetroInd4.0IoT54413.2022.9831741.
A. Malekzadeh, A. Zare, M. Yaghoobi, H. R. Kobravi, and R. Alizadehsani, “Epileptic seizures detection in EEG signals using fusion handcrafted and deep learning features,” Sensors, vol. 21, no. 22, pp. 7710, MDPI, 2021, https://doi.org/10.3390/s21227710.
S. Koelstra, C. Mühl, M. Soleymani, J. S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, “DEAP: A database for emotion analysis using physiological signals,” IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18–31, 2011, https://doi.org/10.1109/T-AFFC.2011.15.
S. Katsigiannis and N. Ramzan, “DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices," IEEE Journal of Biomedical Health and Informatics, vol. 22, no.1, pp. 98-107, IEEE, 2017, https://doi.org/10.1109/JBHI.2017.2688239.
K. Cabello-Solorzano, I. O. de Araujo, M. Peña, L. Correia, and A. J. Tallón-Ballesteros, “The impact of data normalization on the accuracy of machine learning algorithms: A comparative analysis,” in 18th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2023), Lecture Notes in Networks and Systems, vol. 750, 2023, pp. 344-353, Springer, https://doi.org/10.1007/978-3-031-42536-3_33.
M. M. Bradley and P. J. Lang, “Measuring emotion: The self-assessment manikin and the semantic differential,” Journal of Behavior Therapy and Experimental Psychiatry, vol. 25, no. 1, pp. 49–59, Elsevier, 1994, https://doi.org/10.1016/0005-7916(94)90063-9.
R. Nawaz, K. H. Cheah, H. Nisar, and V. V. Yap, “Comparison of different feature extraction methods for EEG-based emotion recognition,” Biocybernetics and Biomedical Engineering, vol. 40, no. 3, pp. 910–926, Elsevier, 2020, https://doi.org/10.1016/j.bbe.2020.04.005.
R. Jenke, A. Peer, and M. Buss, “Feature extraction and selection for emotion recognition from EEG,” IEEE Transactions on Affective Computing, vol. 5, no. 3, pp. 327–339, IEEE, 2014, https://doi.org/10.1109/TAFFC.2014.2339834.
I. Hussain, M. A. Hossain, R. Jany, M. A. Bari, M. Uddin, A. R. M. Kamal, Y. Ku, and J. S. Kim, “Quantitative of evaluation of EEG biomarkers for prediction of sleep stages,” Sensors, vol. 22, no. 8, pp. 3079, MDPI, 2022, https://doi.org/10.3390/s22083079.
M. M. Rahman, A. K. Sarkar, M. A. Hossain, and M. A. Moni, “EEG-based emotion analysis using non-linear features and ensemble learning approaches,” Expert Systems with Applications, vol. 207, pp. 118025, Elsevier, 2022, https://doi.org/10.1016/j.eswa.2022.118025.
H. Peng, F. Long, and C. Ding, “Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1226–1238, IEEE, 200,. https://doi.org/10.1109/TPAMI.2005.159.
N. Gupta, C. K. Verma, et al, “Convolutional Neural Networks: Architecture, Feature Learning, and Evolution,” SSRN, 2025.
L. Alzubaidi, J. Zhang, A.J. Humaidi et al., “Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions,” Journal of Big Data, vol. 8, no. 1, pp. 53, Springer, 2021, https://doi.org/10.1186/s40537-021-00444-8.
T. Talaei Khoei, H. O. Slimane, and N. Kaabouch, “Deep learning: Systematic review, models, challenges, and research directions,” Neural Computing and Applications, vol. 35, no. 31, pp. 23103-23124, 2023, https://doi.org/10.1007/s00521-023-08957-4.
M. Kołodziej, A. Majkowski, P. Tarnowski, and R. J. Rak, “Recognition of visually induced emotions based on electroencephalography,” in 2015 IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 2, pp. 701–704, , IEEE, 2015, https://doi.org/10.1109/IDAACS.2015.7341394.
E. J. Choi and D. K. Kim, “Arousal and valence classification model based on long short-term memory and DEAP data for mental healthcare management,” Healthcare Informatics Research, vol. 24, no. 4, pp. 309–316, Korean Socity of medical Info., 2018, https://doi.org/10.4258/hir.2018.24.4.309.
J. A. Russell, “A circumplex model of affect,” Journal of Personality and Social Psychology, vol. 39, no. 6, pp. 1161, American Psychological Association, 1980, https://psycnet.apa.org/doi/10.1037/h0077714.
C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. https://doi.org/10.1007/BF00994018.
D. M. Powers, “Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation,” arXiv preprint arXiv: 2010.16061, 2020.
K. Devi and B. Deka, “EEG-based emotion classification with per class feature importance using multi-domain features,” in Proceedings of the 2023 International Conference on Computational Intelligence, Networks and Security (ICCINS), IEEE, 2023, pp.1-6, https://doi.org/10.1109/ICCINS58907.2023.10450043.
S. Lundberg, “SHAP (SHapley Additive exPlanations),” GitHub repository. Available: https://github.com/slundberg/shap, 2019.
S. U. Amin, M. Alsulaiman, G. Muhammad, M. A. Bencherif, and M. S. Hossain, “Multilevel weighted feature fusion using convolutional neural networks for EEG motor imagery classification,” IEEE Access, vol. 7, pp. 18940-18950, IEEE, 2019, https://doi.org/10.1109/ACCESS.2019.2895688.
E. Olejarczyk and W. Jernajczyk, “Graph-based analysis of brain connectivity in schizophrenia,” PLOS ONE, vol. 12, no.11, pp. e0188629, Public Library of Science San Francisco, CA USA 2017, https://doi.org/10.1371/journal.pone.0188629.
K. Das and R. B. Pachori, “Schizophrenia detection technique using multivariate iterative filtering and multichannel EEG signals,” Biomedical Signal Processing and Control, vol. 67, pp. 102525, Elsevier, 2021, https://doi.org/10.1016/j.bspc.2021.102525.
K. H. Brodersen, C. S. Ong, K. E. Stephan, and J. M. Buhmann, “The balanced accuracy and its posterior distribution,” in Proc. 20th International Conference on Pattern Recognition (ICPR), 2010, pp. 3121–3124, IEEE, https://doi.org/10.1109/ICPR.2010.764.
J. Rydén and S. E. Alm, “The effect of interaction and rounding error in two-way ANOVA: Example of impact on testing for normality,” Journal of Applied Statistics, vol. 37, no. 10, pp. 1695–1701, Taylor & Francis, 2010, https://doi.org/10.1080/02664760903143925.
Y. Song,Q. Zheng, B. Liu, X. Gao, “EEG conformer: Convolutional transformer for EEG decoding and visualization,” IEEE Trans. on Neural Syst. and Rehabit. Eng, vol. 31, pp. 710-719, 2022, IEEE, https://doi.org/10.1109/TNSRE.2022.3230250.
X. Chen, X. Teng, H. Chen, Y. Pan, P. Geyer, “Towards reliable signal decoding for Electroencephalogram: A benchmark study to EEGNeX,” Biomedical Signal Processing and Control, vol. 87, pp. 105475, 2024, Elsevier, https://doi.org/10.1016/j.bspc.2023.105475.
Copyright (c) 2026 Kabita Devia, Debarun Chakraborty
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlikel 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
.png)
.png)
.png)
.png)
.png)
