Application and algorithm optimization of music emotion recognition in piano performance evaluation
Vol 9, Issue 4, 2024, Article identifier:
VIEWS - 384 (Abstract) 179 (PDF)
Abstract
In the current research, we integrate distinct learning modalities—Curriculum Learning (CL) and Reinforcement Learning (RL)—in an attempt to develop and optimize Music Emotion Recognition (MER) in piano performance. Classical approaches have never been successful when applied in the field of determining the degree of emotion in the music of the piano, owing to the substantial complexity required. Addressing this particular issue is the primary motivation for the present endeavour. In an approach that’s comparable to how human beings acquire information, it trains the RL agent CL in phases; such an approach improves the student’s learning model in understanding the diverse emotions expressed by musical compositions. A higher rating of performance can be achieved after learning the model to recognize emotions more effectively and precisely. A set of piano melodies with emotional content notes has been included in the EMOPIA repository for use when conducting the process of evaluation. In order to benchmark the proposed approach with different models, parameters including R2, Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) were deployed. Studies indicate that the recommended approach accurately recognizes the emotions expressed by piano-playing music. In challenging tasks like MER, the significance of implementing the CL paradigm with the RL has been emphasized using the outcomes mentioned earlier.
Keywords
Full Text:
PDFReferences
1. Cui Y. Vocal music performance evaluation system based on neural network and its application in piano teaching. Revista Ibérica de Sistemas e Tecnologias de Informação. 2023, E55: 451-464.
2. Chang X, Peng L. Evaluation Strategy of the Piano Performance by the Deep Learning Long Short-Term Memory Network. Wireless Communications and Mobile Computing. 2022, 2022: 1-10. doi: 10.1155/2022/6727429
3. Wang W, Pan J, Yi H, et al. Audio-Based Piano Performance Evaluation for Beginners With Convolutional Neural Network and Attention Mechanism. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021, 29: 1119-1133. doi: 10.1109/taslp.2021.3061267
4. Rajesh S, Nalini NJ. Musical instrument emotion recognition using deep recurrent neural network. Procedia Computer Science. 2020, 167: 16-25. doi: 10.1016/j.procs.2020.03.178
5. Chen Q. Intelligent system of piano performance evaluation framework based on multi-dimensional audio recognition algorithm. In: Proceedings of the 2022 6th International Conference on Trends in Electronics and Informatics (ICOEI); 28–30 April 2022; Tirunelveli, India. pp. 82–85. doi: 10.1109/icoei53556.2022.9777178
6. Yu-Chun M, Koong LHC. A study of the affective tutoring system for music appreciation curriculum at the junior high school level. In: Proceedings of the 2016 International Conference on Educational Innovation through Technology (EITT); 22–24 September 2016; Tainan, Taiwan. pp. 204–207. doi: 10.1109/eitt.2016.47
7. Zhang Z, Han J, Coutinho E, et al. Dynamic Difficulty Awareness Training for Continuous Emotion Prediction. IEEE Transactions on Multimedia. 2019, 21(5): 1289-1301. doi: 10.1109/tmm.2018.2871949
8. Wang Y, Sun S. Emotion recognition for internet music by multiple classifiers. In: Proceedings of the 2019 IEEE/ACIS 18th International Conference on Computer and Information Science (ICIS); 17–19 June 2019; Beijing, China. pp. 262–265. doi: 10.1109/icis46139.2019.8940288
9. Yang S, Reed CN, Chew E, et al. Examining Emotion Perception Agreement in Live Music Performance. IEEE Transactions on Affective Computing. 2023, 14(2): 1442-1460. doi: 10.1109/taffc.2021.3093787
10. Gao Z, Qiu L, Qi P, Sun Y. A novel music emotion recognition model for scratch-generated music. In: Proceedings of the 2020 International Wireless Communications and Mobile Computing (IWCMC); 15–19 June 2020; Limassol, Cyprus. pp. 1794–1799. doi: 10.1109/iwcmc48107.2020.9148471
11. Zhang K, Wu X, Tang R, et al. The JinYue database for huqin music emotion, scene and imagery recognition. In: Proceedings of the 2021 IEEE 4th International Conference on Multimedia Information Processing and Retrieval (MIPR); 8–10 September 2021; Tokyo, Japan. pp. 314–319. doi: 10.1109/mipr51284.2021.00059
12. Du P, Li X, Gao Y. Dynamic music emotion recognition based on CNN-BiLSTM. In: Proceedings of the 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC); 12–14 June 2020; Chongqing, China. pp. 1372–1376. doi: 10.1109/itoec49072.2020.9141729
13. Wang H, Zhong W, Ma L, et al. Emotional quality evaluation for generated music based on emotion recognition model. In: Proceedings of the 2022 IEEE International Conference on Multimedia and Expo Workshops (ICMEW); 18–22 July; Taipei City, Taiwan. pp. 1–6. doi: 10.1109/icmew56448.2022.9859459
14. Kumar S, Rani S, Jain A, et al. Face Spoofing, Age, Gender and Facial Expression Recognition Using Advance Neural Network Architecture-Based Biometric System. Sensors. 2022, 22(14): 5160. doi: 10.3390/s22145160
15. Alnuaim AA, Zakariah M, Alhadlaq A, et al. Human-Computer Interaction with Detection of Speaker Emotions Using Convolution Neural Networks. Computational Intelligence and Neuroscience. 2022, 2022: 1-16. doi: 10.1155/2022/7463091
16. Kimmatkar NV, Babu BV. Novel Approach for Emotion Detection and Stabilizing Mental State by Using Machine Learning Techniques. Computers. 2021, 10(3): 37. doi: 10.3390/computers10030037
17. Balajee RM, Mohapatra H, Deepak V, et al. Requirements identification on automated medical care with appropriate machine learning techniques. In: Proceedings of the 2021 6th International Conference on Inventive Computation Technologies (ICICT); 20–22 January 2021; Coimbatore, India. pp. 836–840. doi: 10.1109/icict50816.2021.9358683
18. Mannepalli K, Sastry PN, Suman M. Emotion recognition in speech signals using optimization based multi-SVNN classifier. Journal of King Saud University - Computer and Information Sciences. 2022, 34(2): 384-397. doi: 10.1016/j.jksuci.2018.11.012
19. Balajee R. M., Mohapatra H., Deepak V., Babu D. V., Requirements identification on automated medical care with appropriate machine learning techniques. In: Proceedings of the 2021 6th International Conference on Inventive Computation Technologies (ICICT); 20–22 January 2021; Coimbatore, India. pp. 836–840. doi: 10.1109/ICICT50816.2021.9358683
20. Sekhar Ch, Rao MS, Nayani ASK, Bhattacharyya D. Emotion recognition through human conversation using machine learning techniques. In: Bhattacharyya D, Thirupathi Rao N. (editors). Machine Intelligence and Soft Computing: Proceedings of ICMISC 2020. Volume 1280. pp. 113–122. doi: 10.1007/978-981-15-9516-5_10
21. Durga BK, Rajesh V. A ResNet deep learning based facial recognition design for future multimedia applications. Computers and Electrical Engineering. 2022, 104: 108384. doi: 10.1016/j.compeleceng.2022.108384
22. Ashok Kumar PM, Maddala JB, Martin Sagayam K. Enhanced Facial Emotion Recognition by Optimal Descriptor Selection with Neural Network. IETE Journal of Research. 2021, 69(5): 2595-2614. doi: 10.1080/03772063.2021.1902868
23. Bharti SK, Varadhaganapathy S, Gupta RK, et al. Text-Based Emotion Recognition Using Deep Learning Approach. Computational Intelligence and Neuroscience. 2022, 2022: 1-8. doi: 10.1155/2022/2645381
24. Thirumuru R, Gurugubelli K, Vuppala AK. Novel feature representation using single frequency filtering and nonlinear energy operator for speech emotion recognition. Digital Signal Processing. 2022, 120: 103293. doi: 10.1016/j.dsp.2021.103293
25. Kumar S, Haq M, Jain A, et al. Multilayer Neural Network Based Speech Emotion Recognition for Smart Assistance. Computers, Materials & Continua. 2023, 74(1): 1523-1540. doi: 10.32604/cmc.2023.028631
26. Srinivas PVVS, Mishra P. A novel framework for facial emotion recognition with noisy and de noisy techniques applied in data pre-processing. International Journal of System Assurance Engineering and Management. 2022. doi: 10.1007/s13198-022-01737-8
27. Mishra P, Srinivas PVVS. Facial emotion recognition using deep convolutional neural network and smoothing, mixture filters applied during preprocessing stage. IAES International Journal of Artificial Intelligence (IJ-AI). 2021, 10(4): 889. doi: 10.11591/ijai.v10.i4.pp889-900
28. Setiawan R, Devadass MMV, Rajan R, et al. IoT Based Virtual E-Learning System for Sustainable Development of Smart Cities. Journal of Grid Computing. 2022, 20(3). doi: 10.1007/s10723-022-09616-z
29. Irrinki MK. Learning Through ICT Role of Indian Higher Education Platforms During Pandemic. Library Philosophy and Practice. 2021.
30. Lakshmi AJ, Kumar A, Kumar MS, et al. Artificial intelligence in steering the digital transformation of collaborative technical education. The Journal of High Technology Management Research. 2023, 34(2): 100467. doi: 10.1016/j.hitech.2023.100467
31. Madhavi E, Lavanya Sivapurapu, Vijayakumar Koppula, et al. B. Esther Rani. Developing Learners’ English-Speaking Skills using ICT and AI Tools. Journal of Advanced Research in Applied Sciences and Engineering Technology. 2023, 32(2): 142-153. doi: 10.37934/araset.32.2.142153
32. Nanduri VNPSS, Sagiri C, Manasa SSS, et al. A Review of multi-modal speech emotion recognition and various techniques used to solve emotion recognition on speech data. In: Proceedings of the 2023 5th International Conference on Inventive Research in Computing Applications (ICIRCA). 3–5 August 2023; Coimbatore, India. pp. 577–582. doi: 10.1109/icirca57980.2023.10220691
33. Srinivas P, Khamar SN, Borusu N, et al. Identification of Facial Emotions in Hitech Modern Era. In: Proceedings of the 2023 2nd International Conference on Edge Computing and Applications (ICECAA); 19–21 July 2023; Namakkal, India. pp. 1202–1208. doi: 10.1109/icecaa58104.2023.10212285
34. Sugumar R, Sharma S, Kiran PBN, et al. Novel method for detection of stress in employees using hybrid deep learning models. In: Proceedings of the 2023 8th International Conference on Communication and Electronics Systems (ICCES); 1–3 June 2023; Coimbatore, India. pp. 984–989. doi: 10.1109/ICCES57224.2023.10192609
35. Kuchibhotla S, Dogga SS, Vinay Thota NVSLG, et al. Depression detection from speech emotions using MFCC based recurrent neural network. In: Proceedings of the 2023 2nd International Conference on Vision Towards Emerging Trends in Communication and Networking Technologies (ViTECoN). 5–6 May 2023; Vellore, India. pp. 1–5. doi: 10.1109/vitecon58111.2023.10157779
36. Chintalapudi KS, Patan IAK, Sontineni HV, et al. Speech emotion recognition using deep learning. In: Proceedings of the 2023 International Conference on Computer Communication and Informatics (ICCCI); 23–25 January 2023; Coimbatore, India. pp. 1–5. doi: 10.1109/iccci56745.2023.10128612
37. Fernandes Dimlo UM, Bhanarkar P, Jayalakshmi V, et al. Innovative method for face emotion recognition using hybrid deep neural networks. In: Proceedings of the 2023 7th International Conference on Trends in Electronics and Informatics (ICOEI); 11–13 April 2023; Tirunelveli, India. pp. 876–881. doi: 10.1109/icoei56765.2023.10126007
38. Nallapu SK, Boddukuri VB, Ganesh DVVALS, et al. Intelligent video analytics & facial emotion recognition using artificial intelligence. In: Proceedings of the 2023 Second International Conference on Electronics and Renewable Systems (ICEARS); 2–4 March 2023; Tuticorin, India. pp. 896–900. doi: 10.1109/icears56392.2023.10084928
DOI: https://doi.org/10.54517/esp.v9i4.2344
(384 Abstract Views, 179 PDF Downloads)
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 Yao Zhang, Delin Cai, Dongmei Zhang
License URL: https://creativecommons.org/licenses/by/4.0/