Artificial Intelligence in Speech Emotion Detection: Trends, Challenges, and Future Directions
DOI:
https://doi.org/10.64229/x1jp0z91Keywords:
Speech Emotion Detection (SED), Deep Learning, Feature Extraction, Explainable AI, Multimodal Emotion Recognition, Human–Computer Interaction, Real-time Emotion Detection, Ethical AIAbstract
Speech Emotion Detection (SED) has become a pivotal component in the development of emotionally aware artificial intelligence systems. This paper presents a comprehensive review of recent advancements in the field, focusing on signal processing techniques, machine learning and deep learning approaches, real-time implementation, and multimodal integration. The study highlights the critical role of feature extraction and classification methods in improving emotion recognition accuracy and robustness. Additionally, it discusses emerging trends such as personalization, explainable AI (XAI), and adaptation to noisy and culturally diverse environments. Ethical considerations and legal implications surrounding emotion-aware systems are examined, along with practical applications in healthcare, education, customer support, and entertainment. The review concludes by outlining the unresolved challenges and proposing future research directions to bridge existing gaps and enable more human-centric and trustworthy emotion recognition technologies.
References
[1]N. Sundarprasad, "SPEECH EMOTION DETECTION USING MACHINE LEARNING TECHNIQUES," 2018.
[2]N. Hajarolasvadi and H. Demirel, "3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms," 2019.
[3]I. Pulatov, R. Oteniyazov, F. Makhmudov, and Y. I. Cho, "Enhancing Speech Emotion Recognition Using Dual Feature Extraction Encoders," 2023.
[4]H. Binali, C. Wu, and V. Potdar, "Computational Approaches for Emotion Detection in Text," 2010.
[5]E. Martinelli, A. Mencattini, E. Daprati, and C. Di Natale, "Strength Is in Numbers: Can Concordant Artificial Listeners Improve Prediction of Emotion from Speech?," 2016.
[6]M. Jain, S. Narayan, P. Balaji, B. K P et al., "Speech Emotion Recognition using Support Vector Machine," 2020.
[7]G. Sahu, "Multimodal Speech Emotion Recognition and Ambiguity Resolution," 2019.
[8]M. Kamruzzaman Sarker, K. Md. Rokibul Alam, and M. Arifuzzaman, "Emotion Recognition from Speech based on Relevant Feature and Majority Voting," 2018.
[9]E. Togootogtokh and C. Klasen, "DeepEMO: Deep Learning for Speech Emotion Recognition," 2021.
[10]C. W. Huang and S. S. Narayanan, "Characterizing Types of Convolution in Deep Convolutional Recurrent Neural Networks for Robust Speech Emotion Recognition," 2017.
[11]B. Abu-Salih, M. Alhabashneh, D. Zhu, A. Awajan et al., "Emotion detection of social data: APIs comparative study," 2022.
[12]S. Lai, X. Hu, H. Xu, Z. Ren et al., "Multimodal Sentiment Analysis: A Survey," 2023.
[13]A. Birhala, C. Nicolae Ristea, A. Radoi, and L. Cristian Dutu, "Temporal aggregation of audio-visual modalities for emotion recognition," 2020.
[14]G. Costantini, E. Parada-Cabaleiro, D. Casali, and V. Cesarini, "The Emotion Probe: On the Universality of Cross-Linguistic and Cross-Gender Speech Emotion Recognition via Machine Learning," 2022.
[15]D. Cevher, S. Zepf, and R. Klinger, "Towards Multimodal Emotion Recognition in German Speech Events in Cars using Transfer Learning," 2019.
[16]S. Binti Lebai Lutfi, F. Fernández Martínez, J. Manuel Lucas Cuesta, L. López Lebón et al., "A Satisfaction-based Model for Affect Recognition from Conversational Features in Spoken Dialog Systems," 2013.
[17]S. Latif, H. Shehbaz Ali, M. Usama, R. Rana et al., "AI-Based Emotion Recognition: Promise, Peril, and Prescriptions for Prosocial Path," 2022.
[18]D. C. Ong, "An Ethical Framework for Guiding the Development of Affectively-Aware Artificial Intelligence," 2021.
[19]M. Milling, F. B. Pokorny, K. D. Bartl-Pokorny, and B. W. Schuller, "Is Speech the New Blood? Recent Progress in AI-Based Disease Detection From Audio in a Nutshell," 2022.
[20]P. Fung, D. Bertero, Y. Wan, A. Dey et al., "Towards Empathetic Human-Robot Interactions," 2016.
[21]A. Hauselmann, A. M. Sears, L. Zard, and E. Fosch-Villaronga, "EU law and emotion data," 2023.
[22]A. G. Sabea, M. J. Kadhim, A. F. Neamah, and M. I. Mahdi, “Enhancing medical image analysis with CNN and MobileNet: A particle swarm optimization approach,” Journal of Information Systems Engineering and Management, vol. 10, no. 13s, pp. 28–40, Feb. 2025.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Sinan Adnan Diwan (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
