nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

Bi-Model Emotional AI for Audio-Visual Human Emotion Detection Using Hybrid Deep Learning Model

verfasst von : Anandakumar Haldorai, Babitha Lincy R, Suriya Murugan, Minu Balakrishnan

Erschienen in: Artificial Intelligence for Sustainable Development

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The present attention of computer vision study is on AI emotion identification, which comprises the automatic acknowledgment of facial terminologies of feeling and the evaluation of sentiment in visual database. In order for artificially intelligent systems with visual capabilities to comprehend human interactions, the study of human–machine interaction is essential. Artificial emotional intelligence, sometimes referred to as affective computing and emotional AI, is a subfield of artificial intelligence that concentrates on the comprehension, examination, and replication of human emotions. Its goal is to advance the sincerity and organic nature of interactions between people and robots. Textual content, voice tone, facial expressions, and gestures are just a few of the cues that emotional AI uses to understand people’s emotions and alter its answers accordingly. Using computer vision technology, Visual Emotion AI analyzes facial expressions in photos and videos to determine a person’s emotional state. This study uses both audio and visual inputs to investigate the recognition of emotions using artificial intelligence.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel AI-Based Effective Communication in Software-Defined VANET: A Study

Nächstes Kapitel An End-to-End Offline Handwritten Tamil Text Identification Using Modified RAdam Optimizer with Effective Post-processing Techniques

J. Deng and F. Ren, "Multi-Label Emotion Detection via Emotion-Specified Feature Extraction and Emotion Correlation Learning," in IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 475–486, 1 Jan.-March 2023, doi: https://doi.org/10.1109/TAFFC.2020.3034215.

H. Kim, J. Ben-othman, L. Mokdad and P. Bellavista, "A Virtual Emotion Detection Architecture With Two-Way Enabled Delay Bound toward Evolutional Emotion-Based IoT Services," in IEEE Transactions on Mobile Computing, vol. 21, no. 4, pp. 1172–1181, 1 April 2022, doi: https://doi.org/10.1109/TMC.2020.3024059.

Y. Yu et al., "Cloud-Edge Collaborative Depression Detection Using Negative Emotion Recognition and Cross-Scale Facial Feature Analysis," in IEEE Transactions on Industrial Informatics, vol. 19, no. 3, pp. 3088–3098, March 2023, doi: https://doi.org/10.1109/TII.2022.3163512.

F. Anzum and M. L. Gavrilova, "Emotion Detection From Micro-Blogs Using Novel Input Representation," in IEEE Access, vol. 11, pp. 19512–19522, 2023, doi: https://doi.org/10.1109/ACCESS.2023.3248506.

C. Suman, R. Chaudhari, S. Saha, S. Kumar and P. Bhattacharyya, "Investigations in Emotion Aware Multimodal Gender Prediction Systems From Social Media Data," in IEEE Transactions on Computational Social Systems, vol. 10, no. 2, pp. 470-479, April 2023, doi: https://doi.org/10.1109/TCSS.2022.3158605.

Z. Xu and S. Wang, "Emotional Attention Detection and Correlation Exploration for Image Emotion Distribution Learning," in IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 357-369, 1 Jan.-March 2023, doi: https://doi.org/10.1109/TAFFC.2021.3071131.

N. Aslam, F. Rustam, E. Lee, P. B. Washington and I. Ashraf, "Sentiment Analysis and Emotion Detection on Cryptocurrency Related Tweets Using Ensemble LSTM-GRU Model," in IEEE Access, vol. 10, pp. 39313–39324, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3165621.

F. Alrowais et al., "Modified Earthworm Optimization With Deep Learning Assisted Emotion Recognition for Human Computer Interface," in IEEE Access, vol. 11, pp. 35089-35096, 2023, doi: https://doi.org/10.1109/ACCESS.2023.3264260.

E. A. Veltmeijer, C. Gerritsen and K. V. Hindriks, "Automatic Emotion Recognition for Groups: A Review," in IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 89–107, 1 Jan.–March 2023, doi: https://doi.org/10.1109/TAFFC.2021.3065726.

10.

S. Lee, S. Lee and H. Kim, "Differential Security Barriers for Virtual Emotion Detection in Maritime Transportation Stations With Cooperative Mobile Robots and UAVs," in IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 2, pp. 2461–2471, Feb. 2023, doi: https://doi.org/10.1109/TITS.2022.3172668.

11.

L. Chen, M. Zhou, W. Su, M. Wu, J. She, and K. Hirota, “Softmax regression based deep sparse autoencoder network for facial emotion recognition in human-robot interaction,” Information Sciences, vol. 428, pp. 49–61, 2018.MathSciNetCrossRef

12.

A. Hassouneh, A. M. Mutawa, and M. Murugappan, “Development of a real-time emotion recognition system using facial expressions and EEG based on machine learning and deep neural network methods,” Informatics in Medicine Unlocked, vol. 20, Article ID 100372, 2020.

13.

O. Ekundayo and S. Viriri, “Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation,” PeerJ Computer Science, vol. 7, p. e736, 2021.

14.

K. Jayanthi and S. Mohan, “An integrated framework for emotion recognition using speech and static images with deep classifier fusion approach,” International Journal of Information Technology, pp. 1–11, 2022.

15.

P. Babajee, G. Suddul, S. Armoogum, and R. Foogooa, “Identifying human emotions from facial expressions with deep learning,” in Proceedings of the 2020 Zooming Innovation in Consumer Technologies Conference (ZINC), pp. 36–39, IEEE, Novi Sad, Serbia, May 2020.

16.

C. Tan, M. Šarlija, and N. Kasabov, “NeuroSense: short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns,” Neurocomputing, vol. 434, pp. 137–148, 2021.CrossRef

17.

M. S. Hossain and G. Muhammad, “An Audio-Visual Emotion Recognition System Using Deep Learning Fusion for a Cognitive Wireless Framework,” IEEE Wireless Communications, vol. 26, no. 3, pp. 62–68, Jun. 2019, doi: https://doi.org/10.1109/mwc.2019.1800419.

18.

Yining Chen, “The Past, Present, and Future of Enterprise Resource Planning”, Journal of Enterprise and Business Intelligence, vol.3, no.2, pp. 065–074, April 2023. doi: https://doi.org/10.53759/5181/JEBI202303007.

19.

B. Jackson, S. R, B. Balachander, A. Haldorai, V. S., and A. N. A. Sahaya, “Modified Design Structure Of A Metamaterial Microstrip Patch Array Antenna For Rf Energy Optimization,” Materiali in tehnologije, vol. 56, no. 2, Apr. 2022, doi: 10.17222/mit.2022.384

20.

Sim Sze Yin and Yoni Danieli, “An Evaluation of Smart Livestock Feeding Strategies”, Journal of Robotics Spectrum, vol.1, pp. 066–077, 2023. doi: https://doi.org/10.53759/9852/JRS202301007.

21.

A. Kumar, K. Abhishek, X. Liu, and A. Haldorai, “An Efficient Privacy-Preserving ID Centric Authentication in IoT Based Cloud Servers for Sustainable Smart Cities,” Wireless Personal Communications, vol. 117, no. 4, pp. 3229–3253, Nov. 2020, doi: https://doi.org/10.1007/s11277-020-07979-8.

22.

Kazuo Adachi and Ryo Sato, “Customer Relationship Management in an Era of Digital Disruption and Business Model Evolution”, Journal of Enterprise and Business Intelligence, vol.3, no.1, pp. 001–011, January 2023. doi: https://doi.org/10.53759/5181/JEBI202303001.

23.

S. Ntalampiras, “Speech emotion recognition via learning analogies,” Pattern Recognition Letters, vol. 144, pp. 21–26, Apr. 2021, doi: https://doi.org/10.1016/j.patrec.2021.01.018.

24.

M. Bejani, D. Gharavian, and N. M. Charkari, “Audiovisual EmotionRecognition using ANOVA Feature Selection Method and Multi-ClassifierNeural Networks,” Neural Computing and Applications, vol. 24, no. 2,pp. 399–412, 2014.CrossRef

25.

N. E. D. Elmadany, Y. He, and L. Guan, “Multiview Learning via DeepDiscriminative Canonical Correlation Analysis,” in IEEE InternationalConference on Acoustics, Speech and Signal Processing. IEEE, 2016,pp. 2409–2413.

26.

S. Zhang, S. Zhang, T. Huang, and W. Gao, “Multimodal DeepConvolutional Neural Network for Audio-Visual Emotion Recognition,“in ACM on International Conference on Multimedia Retrieval. ACM,2016, pp. 281–284.

27.

W. Mellouk and W. Handouzi, “Facial emotion recognition using deep learning: review and insights,” Procedia Computer Science, vol. 175, pp. 689–694, 2020, doi: https://doi.org/10.1016/j.procs.2020.07.101.CrossRef

28.

R. Gross, S. Baker, I. Matthews, and T. Kanade, “Face Recognition Across Pose and Illumination,” Handbook of Face Recognition, pp. 193–216, doi: 10.1007/0-387-27257-7_10.

29.

A. Dhall, R. Goecke, S. Lucey, and T. Gedeon, “Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark,” 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Nov. 2011, doi: https://doi.org/10.1109/iccvw.2011.6130508.

30.

P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression,” 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition – Workshops, Jun. 2010, doi: https://doi.org/10.1109/cvprw.2010.5543262

31.

M. F. Valstar, B. Jiang, M. Mehu, M. Pantic, and K. Scherer, “The first facial expression recognition and analysis challenge,” Face and Gesture 2011, Mar. 2011, doi: https://doi.org/10.1109/fg.2011.5771374.

32.

I. J. Goodfellow et al., “Challenges in representation learning: A report on three machine learning contests,” Neural Networks, vol. 64, pp. 59–63, Apr. 2015, doi: https://doi.org/10.1016/j.neunet.2014.09.005.

33.

Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang, and M. J. Rosato, “A 3D Facial Expression Database For Facial Behavior Research,” 7th International Conference on Automatic Face and Gesture Recognition (FGR06), doi: 10.1109/fgr.2006.6.

34.

W.-J. Yan et al., “CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation,” PLoS ONE, vol. 9, no. 1, p. e86041, Jan. 2014, doi: https://doi.org/10.1371/journal.pone.0086041.

35.

G. Zhao, X. Huang, M. Taini, S. Z. Li, and M. Pietikäinen, “Facial expression recognition from near-infrared videos,” Image and Vision Computing, vol. 29, no. 9, pp. 607–619, Aug. 2011, doi: https://doi.org/10.1016/j.imavis.2011.07.002.

36.

A. Mollahosseini, B. Hasani, and M. H. Mahoor, “AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild,” IEEE Transactions on Affective Computing, vol. 10, no. 1, pp. 18–31, Jan. 2019, doi: https://doi.org/10.1109/taffc.2017.2740923.

37.

S. Li, W. Deng, and J. Du, “Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, doi: https://doi.org/10.1109/cvpr.2017.277.

38.

O. Langner, R. Dotsch, G. Bijlstra, D. H. J. Wigboldus, S. T. Hawk, and A. van Knippenberg, “Presentation and validation of the Radboud Faces Database,” Cognition & Emotion, vol. 24, no. 8, pp. 1377–1388, Dec. 2010, doi: https://doi.org/10.1080/02699930903485076.

39.

A. V. Savchenko, L. V. Savchenko and I. Makarov, "Classifying Emotions and Engagement in Online Learning Based on a Single Facial Expression Recognition Neural Network," in IEEE Transactions on Affective Computing, vol. 13, no. 4, pp. 2132–2143, 1 Oct.-Dec. 2022, doi: https://doi.org/10.1109/TAFFC.2022.3188390.

40.

T. Dar, A. Javed, S. Bourouis, H. S. Hussein and H. Alshazly, "Efficient-SwishNet Based System for Facial Emotion Recognition," in IEEE Access, vol. 10, pp. 71311–71328, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3188730.

41.

F. Huang, X. Li, C. Yuan, S. Zhang, J. Zhang and S. Qiao, "Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis," in IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 9, pp. 4332–4345, Sept. 2022, doi: https://doi.org/10.1109/TNNLS.2021.3056664.

42.

B. García-Martínez, A. Fernández-Caballero, R. Alcaraz and A. Martínez-Rodrigo, "Application of Dispersion Entropy for the Detection of Emotions With Electroencephalographic Signals," in IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 3, pp. 1179–1187, Sept. 2022, doi: https://doi.org/10.1109/TCDS.2021.3099344.

43.

S. Kakuba, A. Poulose and D. S. Han, "Deep Learning-Based Speech Emotion Recognition Using Multi-Level Fusion of Concurrent Features," in IEEE Access, vol. 10, pp. 125538–125551, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3225684.

Titel: Bi-Model Emotional AI for Audio-Visual Human Emotion Detection Using Hybrid Deep Learning Model
verfasst von: Anandakumar Haldorai
Babitha Lincy R
Suriya Murugan
Minu Balakrishnan
Verlag: Springer Nature Switzerland
Buch: Artificial Intelligence for Sustainable Development
Print ISBN: 978-3-031-53971-8

Electronic ISBN: 978-3-031-53972-5

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-3-031-53972-5_15

Neuer Inhalt

Bildnachweise

VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, Die Gewinner und Laudatoren des Sustainability Award in Automotive 2024/© Uli Regenscheit | ATZlive, Search Icon, Banner Hanser, Additiv gefertigte Teile/© Marina_Skoropadskaya | Getty Images | iStock, Warnschild "Land unter"/© Bluedesign / Fotolia, Gardiner von Trapp/© Alpega Group, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, ATZ-Webinar: Prototypenfreie Entwicklung durch Offline- und Driver-in-the-Loop-HiL-Tests /© (c) VI-grade, chassis.tech plus 2023/© [M] ATZlive / TÜV SÜD PRODUCT SERVICE GMBH, adäsion-Webinar-Matinee/© krystiannawrocki_ Getty Images

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.